OTEP: Agent threat detection semantic conventions#5098
Conversation
Proposes a vendor-neutral namespace for recording runtime threat-detection outcomes on AI agent spans. Five attributes: rule_id, ruleset, severity, action, correlation_id. Composable with gen_ai.* without overlap. Follow-up to semantic-conventions-genai#132.
There was a problem hiding this comment.
Pull request overview
This pull request adds a new OTEP proposing an agent.threat.detection.* semantic attribute namespace to record outcomes of runtime threat-detection evaluations on AI agent spans in a vendor-/ruleset-agnostic way.
Changes:
- Introduces a draft OTEP defining five
agent.threat.detection.*attributes (rule ID, ruleset, severity, action, correlation ID). - Describes intended usage patterns (on spans vs. span events / dedicated detection spans) and interaction with existing
gen_ai.*anderror.*conventions. - Documents trade-offs, alternatives, open questions, and example prior art/prototypes.
Comments suppressed due to low confidence (2)
oteps/5098-agent-threat-detection.md:57
- In the “child span” case, saying that
correlation_idlinks to the parent operation is potentially misleading/redundant because the parent/child relationship already provides that linkage within a trace. Consider clarifying thatcorrelation_idis intended to correlate across traces/systems (or to external tickets), and/or that it’s primarily for asynchronous/out-of-band detections where trace relationships aren’t sufficient.
1. **On the existing operation span.** A guardrail wrapped around `execute_tool` sets the attributes directly on the `execute_tool` span. This is the common case when the detection layer is in-process and the operation does not duplicate.
2. **On a span event or a dedicated detection span.** When the detection is asynchronous (for example, an out-of-band scanner that re-reads a tool output) or when multiple rules fire on one operation, each detection is a span event under the operation span, or a child span with the same attributes plus `correlation_id` linking to the parent operation.
oteps/5098-agent-threat-detection.md:50
- The document describes
correlation_idas “not intended for high-cardinality joins inside a tracing backend” but later calls it “the natural join key” to related repair spans. Consider reconciling this guidance (e.g., clarify that correlation is for external systems/offline analysis, or recommend span links for in-backend correlation while keeping the attribute optional/low-impact).
Cardinality is bounded by ruleset size, which is operationally small for production rulesets (today on the order of 10^2 to 10^3). The `correlation_id` is opaque and is not intended for high-cardinality joins inside a tracing backend.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | `agent.threat.detection.rule_id` | string | Conditionally required when a rule matched | Stable identifier of the matched rule, scoped by `ruleset`. Example: `ATR-2026-00081`, `guardrail-pii-7`. | | ||
| | `agent.threat.detection.ruleset` | string | Conditionally required when `rule_id` is set | Namespace plus version of the ruleset. Example: `atr@v2.1.3`, `vendor-x@2026.04`. | | ||
| | `agent.threat.detection.severity` | string enum | Conditionally required when `rule_id` is set | One of `low`, `medium`, `high`, `critical`. Aligned with common security severity scales. | | ||
| | `agent.threat.detection.action` | string enum | Required when a detection layer evaluated the operation | One of `allow`, `block`, `warn`, `ask`. Records the enforcing decision. `ask` covers human-in-the-loop approval flows (see issue #95). | |
|
Please remove named implementation prototypes from this OTEP, or replace them with category-level examples such as open community rulesets, vendor proprietary rulesets, and local policy packs. The convention should not make one ruleset look privileged before the attribute shape is accepted. This follows from the namespace-neutrality discussion on open-telemetry/semantic-conventions-genai#132. The same way |
|
hi @eeee2345! we don't generally use OTEPs for semantic convention work, I'd suggest continuing the discussion in the semantic-conventions-genai repository, or if you think it should be broader, then in the semantic-conventions repository. There are also weekly SIG meetings for both of those repositories (see here and here) which are open to all if you are able to attend and raise the topic there it can often help to get more discussion going. thanks! |
|
Thanks @trask, that makes sense. Closing this and consolidating into semantic-conventions-genai#132, which is the right repo for semantic-convention work and already open. For continuity, the OTEP content here covered To @luckyPipewrench's point about not privileging any one ruleset in the naming — the genai-repo migration will use category-level examples (open community rulesets, vendor proprietary rulesets, local policy packs) rather than naming specific implementations. Will keep the discussion neutral until the attribute shape lands. No action needed on this PR. Closing. |
Per discussion in semantic-conventions-genai#132 with @AgentGymLeader and @luckyPipewrench, this PR opens an OTEP for an agent.threat.detection.* attribute namespace.
Summary
The OTEP introduces five attributes for recording the outcome of runtime threat-detection evaluations on AI agent spans:
Reviewer feedback on issue #132 pushed the namespace from gen_ai.threat.* to agent.threat.detection., on the grounds that some detection layers (for example, MCP firewalls) are agent-aware but not strictly GenAI-API-aware. The OTEP keeps that placement. The attributes are composable with the existing gen_ai. conventions without overlap, and they also apply to non-GenAI agent operations.
The OTEP file is at oteps/NNNN-agent-threat-detection.md and will be renumbered once a PR number is assigned. Word count is approximately 1440.
Scope
This OTEP standardizes the attribute shape only. It does not standardize any specific ruleset. Any detection layer, proprietary or open source, can populate the attributes.
Evidence that detection layers with stable rule IDs are already operating in production
These are listed as concrete examples that the proposed schema fits real deployments. The OTEP itself does not depend on any of them.
Related issues
Open questions called out in the OTEP
Happy to iterate on namespace, enum values, and attribute requirements based on review feedback.